{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d1f20875-c50d-46df-906b-49ec1d0c6002",
   "metadata": {},
   "source": [
    "# 7.7 实现误差反向传播算法"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "c90a18b9-d8e8-431c-856c-35f1e4ed05e1",
   "metadata": {},
   "source": [
    "### 1.任务描述\n",
    "\n",
    "假设有一个两层神经网络，网络的输入层、隐含层和输出层都只有一个神经元。隐含层和输出层的激活函数都是Sigmoid函数，各层的表达式如下。\n",
    "\n",
    "1. 隐含层\n",
    "\n",
    "- 输入：隐含层神经元接收输入值x。\r",
    "- \n",
    "线性计算：通过本层的参数$w_h$（$b_h$和）进行线性计算，得$z_h=w_hx+b_h$：- \n",
    "\r\n",
    "激活：将作为Sigmoid激活函数的输入，计$y_h=\\frac{1}{1+e^{-z_h}}$输- ：\r\n",
    "\r\n",
    "输出：隐含层将激活$y_h$函数的输出作为本层的输出。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59c59bf0-98e0-4fbc-99f8-8b3c8f8a00d7",
   "metadata": {},
   "source": [
    "2. 输出层\n",
    "\n",
    "输入：将隐含层的输出$y_h$作为输出层的输入。\r\n",
    "线性计算：通过本层的参数$w_o$（$b_o$和）进行线性计算，得$z_o=w_oy_h+b_o$到：\r\n",
    "\r\n",
    "激$z_o$活：将作为Sigmoid激活函数的输入，计$y_o=\\frac{1}{1+e-z_o}$算输出：\r\n",
    "\r\n",
    "输出：输出层将激活$y_o$函数的输出作为本层的输出。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f9483f3-656b-4640-805b-3859b766c2f4",
   "metadata": {},
   "source": [
    "假设样本特征x为1，样本标签y为0.8。则隐含层参数初始值：$w_h=0.2,b_h=0.1$；输出层参数初始值：$w_o=0.3,b_o=0.2$，学习率$\\eta=0.5$。\n",
    "\n",
    "求：网络训练，将样本数据x输入网络，通过误差反向传播算法寻找合适的模型参$w_h,b_h,w_o,b_o$，使得网络的输出与样本数据的标签y一致。"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "f5b4fc39-cbcf-432a-bf1e-e75e642d4b87",
   "metadata": {},
   "source": [
    "### 2.知识准备\n",
    "\n",
    "见教程。"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "b624ebee-980f-4c1e-b963-a24ff0b669f6",
   "metadata": {},
   "source": [
    "### 3.任务分析\n",
    "\n",
    "网络学习的步骤如下：\n",
    "\n",
    "1. 设置模型参数初始值\n",
    "\n",
    "隐含层参数初始值：$w_h=0.2,b_h=0.1$，输出层参数初始值：$w_o=0.3,b_o=0.2$，学习率$\\eta=0.5$。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ebf40b2-f4f9-438b-9adb-5f6d2b2de425",
   "metadata": {},
   "source": [
    "2. 正向计算预测值\n",
    "\n",
    "使用模型参数初始值和样本x逐层计算，得到神经网络的预测值，隐含层的输出为：\n",
    "\n",
    "$$y_h=\\frac{1}{1+e^{-(0.2*1+0.1)}}\\approx0.57$$\n",
    "\n",
    "输出层的输出为：\n",
    "\n",
    "$$y_o=\\frac{1}{1+e^{-(0.3*0.57+0.2)}}\\approx0.59$$\n",
    "\n",
    "$y_o$就是网络的预测值，其与样本标签（0.8）差距比较大，需要使用误差调整网络参数，也就是训练网络。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5e0611da-1833-4d41-9bfa-a2afd4ec5b89",
   "metadata": {},
   "source": [
    "3. 计算误差\n",
    "\n",
    "使用平方损失函数计算出预测值和标签之间的误差：\n",
    "\n",
    "$$Loss=\\frac{1}{2}(y-y_o)^2=\\frac{1}{2}(0.8-0.59)^2=0.02205$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3bcb850-2839-4f62-ab09-559128f400fa",
   "metadata": {},
   "source": [
    "4. 误差反向传播\n",
    "\n",
    "对损失函数的梯度信息进行反向传播，同时更新所有层的模型参数。\n",
    "\n",
    "- 更新输出层参数\n",
    "\n",
    "梯度下降法参数更新公式为：\n",
    "\n",
    "$$w_o^{(k+1)}=w_o^{(k)}-\\eta\\frac{\\partial Loss}{\\partial w_o}$$\n",
    "$$b_o^{(k+1)}=b_o^{(k)}-\\eta\\frac{\\partial Loss}{\\partial b_o}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7978e5fe-232b-4934-bd9b-4c7e44158333",
   "metadata": {},
   "source": [
    "(1)求Loss对$w_o$的偏导数\n",
    "\n",
    "在损失函数中，$Loss,y_o,z_o$三个函数之间是嵌套关系，关系式如下：\n",
    "\n",
    "$$Loss=\\frac{1}{2}(y-y_o)^2$$\n",
    "$$y_o=\\frac{1}{1+e^{(-z_o)}}$$\n",
    "$$z_o=w_oy_h+b_o$$\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fb6cd75-22cc-4ca9-bdb1-5bacee5f3e77",
   "metadata": {},
   "source": [
    "要得到损失函数$Loss$对$w_o$的偏导数，可以使用链式求导法则：\n",
    "\n",
    "$$\\frac{\\partial Loss}{\\partial w_o}=\\frac{\\partial Loss}{\\partial y_o}*\\frac{\\partial y_o}{\\partial z_o}*\\frac{\\partial z_o}{\\partial w_o}$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99c75141-f530-42b0-a42e-e75f040e5d6b",
   "metadata": {},
   "source": [
    "分别计算偏导数：\n",
    "\n",
    "$$\\frac{\\partial Loss}{\\partial y_o}=2*\\frac{1}{2}*(y-y_o)*(-1)=-(0.8-0.59)=-0.21$$\n",
    "$$\\frac{\\partial y_o}{\\partial z_o}=\\frac{e^{-z_o}}{(1+e{-z_o)^2}}=y_o*(1-y_o)=0.59*(1-0.59)=0.2419$$\n",
    "$$\\frac{\\partial z_o}{\\partial w_o}=y_h+0=0.57$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02ca29c2-4ac0-44c5-bdf8-3c155e4af469",
   "metadata": {},
   "source": [
    "将这三个偏导数代入Loss对的偏导数，得到：\n",
    "\n",
    "$$\\frac{\\partial Loss}{\\partial w_o}=\\frac{\\partial Loss}{\\partial y_o}*\\frac{\\partial y_o}{\\partial z_o}*\\frac{\\partial z_o}{\\partial w_o}=-0.21*0.2419*0.57=-0.0295543$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75b67751-b870-452b-bf08-6bb269c019cb",
   "metadata": {},
   "source": [
    "(2)求Loss对$b_o$的偏导数\n",
    "\n",
    "$$\\frac{\\partial Loss}{\\partial b_o}=\\frac{\\partial Loss}{\\partial y_o}*\\frac{\\partial y_o}{\\partial z_o}*\\frac{\\partial z_o}{\\partial b_o}=-0.21*0.2419*1=-0.050799$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fcf6398-2ecd-4309-a410-70ee6a6157f9",
   "metadata": {},
   "source": [
    "(3)使用迭代公式更新$w_o$和$b_o$\n",
    "\n",
    "$$w_o^{(1)}=w_o^{(0)}-\\eta\\frac{\\partial Loss}{\\partial w_o}=0.3-0.5*(-0.02895543)=0.314477715$$\n",
    "$$b_o^{(1)}=b_o^{(0)}-\\eta\\frac{\\partial Loss}{\\partial b_o}=0.2-0.5*(-0.050799)=0.2253995$$\n",
    "\n",
    "现在，输出层的模型参数已更新完成。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c5a39b4-3671-47ad-8e9a-d741b21331b0",
   "metadata": {},
   "source": [
    "- 更新隐含层的参数\n",
    "\n",
    "使用梯度下降法计算损失函数对隐含层的参数$w_h$和$b_h$的偏导数。\n",
    "\n",
    "(1)求损失函数对$w_h$的偏导数。\n",
    "\n",
    "损失函数中$Loss$与$y_o,z_o,y_H,z_h$之间是嵌套关系，关系式如下：\n",
    "\n",
    "$$Loss=\\frac{1}{2}(y-y_o)^2$$\n",
    "$$y_o=\\frac{1}{1+e^{(-z_o)}}$$\n",
    "$$z_o=w_oy_h+b_o$$\n",
    "$$y_h=\\frac{1}{1+e{-z_h}}$$\n",
    "$$z_h=w_hx+b_h$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0edb6e11-84f6-4394-a2a6-322f393fde71",
   "metadata": {},
   "source": [
    "要得到损失函数Loss对$w_h$的偏导数，可以使用链式求导法则：\n",
    "\n",
    "$$\\frac{\\partial Loss}{\\partial w_h}=\\frac{\\partial Loss}{\\partial y_o}*\\frac{\\partial y_o}{\\partial z_o}*\\frac{\\partial z_o}{\\partial y_h}*\\frac{\\partial y_h}{\\partial z_h}*\\frac{\\partial z_h}{\\partial w_h}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3d17c7f-04aa-479e-8141-68a4aa28c4ba",
   "metadata": {},
   "source": [
    "分别计算偏导数：\n",
    "\n",
    "$$\\frac{\\partial Loss}{\\partial y_o}=2*\\frac{1}{2}*(y-y_o)*(-1)=-(0.8-0.59)=-0.21$$\n",
    "$$\\frac{\\partial y_o}{\\partial z_o}=\\frac{e^{-z_o}}{(1+e^{-z_o})^2}=y_o(1-y_o)=0.59*(1-0.59)=0.2419$$\n",
    "$$\\frac{\\partial z_o}{\\partial y_h}=w_o=0.3$$\n",
    "$$\\frac{\\partial y_h}{\\partial z_h}=y_h(1-y_h)=0.57*(1-0.57)=0.2451$$\n",
    "$$\\frac{\\partial z_h}{\\partial w_h}=x=1$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd36d0b9-e470-40b7-872a-8cc4fb74699e",
   "metadata": {},
   "source": [
    "将计算结果代入Loss对$w_h$的偏导数，得到：\n",
    "\n",
    "$$\\frac{\\partial Loss}{\\partial w_h}=\\frac{\\partial Loss}{\\partial y_o}*\\frac{\\partial y_o}{\\partial z_o}*\\frac{\\partial z_o}{\\partial y_h}*\\frac{\\partial y_h}{\\partial z_h}*\\frac{\\partial z_h}{\\partial w_h}=-0.21*0.2419*0.3*0.2451*1=-0.00373525$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b6e0111-9391-4426-b1c9-8ba383a7fcd5",
   "metadata": {},
   "source": [
    "(2)求损失函数对$b_h$的偏导数。\n",
    "\n",
    "同样采用链式求导法则求$Loss$对$b_h$的偏导数：\n",
    "\n",
    "\r\n",
    "$$\\frac{\\partial Loss}{\\partial b_h}=\\frac{\\partial Loss}{\\partial y_o}*\\frac{\\partial y_o}{\\partial z_o}*\\frac{\\partial z_o}{\\partial y_h}*\\frac{\\partial y_h}{\\partial z_h}*\\frac{\\partial z_h}{\\partial b_h}=-0.21*0.2419*0.3*0.2451*1=-0.00373525$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "70702635-2ce2-4c5a-ba22-393af1ca5554",
   "metadata": {},
   "source": [
    "(3)更新$w_h$和$b_h$\n",
    "\n",
    "使用迭代公式更新$w_h$和$b_h$，公式如下：\n",
    "\n",
    "$$w_h^{(1)}=w_h^{(0)}-\\eta\\frac{\\partial Loss}{\\partial w_h}=0.2-0.5(-0.00373525)=0.201867625$$\n",
    "$$b_h^{(1)}=b_h^{(0)}-\\eta\\frac{\\partial Loss}{\\partial b_h}=0.1-0.5(-0.00373525)=0.101867625$$\n",
    "\n",
    "至此，对网络中的所有模型参数都更新了，完成了一轮训练。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ff8848b-7a41-4926-bdda-5fbd52fc9c37",
   "metadata": {},
   "source": [
    "5. 迭代\n",
    "\n",
    "使用新的参数逐层正向计算，得到新的预测值，然后与标签值进行比较，计算误差，再逐层反向传播损失函数的梯度信息，更新模型参数，完成下一轮训练。如此循环，直到误差收敛到一个极小的值。"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "435c6090-cfda-4f46-a550-22a368e41e4a",
   "metadata": {},
   "source": [
    "### 4.任务实施\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ec75eb6c-5da3-467d-a471-ca3b47242dd6",
   "metadata": {},
   "source": [
    "执行代码"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "2ae9da58-e339-4d22-9f8d-ca255711d89e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 wh: 0.2 ,bh: 0.1 ,wo: 0.3 ,bo: 0.2 ,yo: 0.59202254 ,loss: 0.021627314\n",
      "10 wh: 0.21779878 ,bh: 0.11779876 ,wo: 0.41937417 ,bo: 0.40649143 ,yo: 0.65724415 ,loss: 0.010189619\n",
      "20 wh: 0.23266931 ,bh: 0.1326693 ,wo: 0.4984085 ,bo: 0.54130167 ,yo: 0.6975226 ,loss: 0.005250812\n",
      "30 wh: 0.24434496 ,bh: 0.14434496 ,wo: 0.55323696 ,bo: 0.6337892 ,yo: 0.7238221 ,loss: 0.002901536\n",
      "40 wh: 0.25343347 ,bh: 0.15343344 ,wo: 0.5928223 ,bo: 0.69999564 ,yo: 0.741904 ,loss: 0.0016875721\n",
      "50 wh: 0.26054475 ,bh: 0.16054471 ,wo: 0.6222935 ,bo: 0.7489627 ,yo: 0.75485367 ,loss: 0.0010190962\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiIAAAFvCAYAAAB3mqAFAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/NK7nSAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA4vklEQVR4nO3deXgUVb7G8W8nIQlbVjYJAQJElDEgA2GCDCCLAy6RQXDh4ohL1BEjOAhXUUFnMbiMoHgDgiAqq8oM4oaggtsYBWKEhAiIBoUgqyGdsIQldf84k4SwhCZ0utLd7+d56umqrj70r+qJ0+9UnTrHYVmWhYiIiIgNAuwuQERERPyXgoiIiIjYRkFEREREbKMgIiIiIrZREBERERHbKIiIiIiIbRRERERExDYKIiIiImKbILsLqM1KS0vZsWMHDRs2xOFw2F2OiIiI17Asi6KiIpo3b05AwJmveyiIVGHHjh3ExsbaXYaIiIjX2rZtGy1atDjjfgWRKjRs2BAwJzEsLMzmakRERLyH0+kkNja2/Lf0TBREqlB2OyYsLExBREREpBrO1rVBnVVFRETENgoiIiIiYhsFEREREbGNgoiIiIjYRkFEREREbKMgIiIiIrZREBERERHbKIiIiIiIbRRERERExDYKIh5kWZCVBTNn2l2JiIhI7aAh3j3ohx/gt7+FwEAYNAiaNrW7IhEREXvpiogHtWsHv/sdHD8O8+fbXY2IiIj9FEQ87NZbzeucOeZWjYiIiD9TEPGwG2+EkBDIyYFvvrG7GhEREXspiHhYZCQMHmzWX3nF1lJERERspyBig7LbMwsWQEmJraWIiIjYSkHEBv37Q/Pm8Ouv8M47dlcjIiJiHwURGwQGwi23mHXdnhEREX+mIGKTstszH3wAv/xiaykiIiK2URCxSfv20L27xhQRERH/piBiI40pIiIi/k5BxEY33gihoZCbC2vX2l2NiIiI5ymI2Cg8HK67zqyr06qIiPgjBRGbnTimyOHDtpYiIiLicQoiNuvbF1q0gP374e237a5GRETEsxREbBYYCCNGmHXdnhEREX+jIFILlAWR5cshP9/eWkRERDxJQaQWiI+HHj2gtBTmzbO7GhEREc9REKklyjqtvvKKxhQRERH/YUsQycnJITExkcjISMaNG4flwi/v4sWLadWqFc2bN2fhwoXl7x8/fpx77rmHsLAw6tWrx5133smxY8fK96enp9O0aVPatGnDypUra+R43OGGG6BuXdi4Eb7+2u5qREREPMPjQaSkpITk5GS6dOnC2rVryc3N5ZWz9NLMyclh+PDhTJgwgeXLlzNx4kQ2bdoEwJNPPklWVhZfffUVX375JUuXLmXOnDkALF++nLFjxzJz5kzmzZtHSkoK+/btq+lDrJawMBgyxKyr06qIiPgLjweRZcuWUVhYyOTJk2nbti1paWnMnj27yjazZs2iT58+pKSkkJCQQGpqKnPnzgVg7969LFiwgA4dOnDppZdy5ZVXkpWVBcD06dMZMWIEgwYN4rLLLmPQoEEsWbKkxo+xuspuzyxaBIcO2VqKiIiIR3g8iKxbt46kpCTq1asHQMeOHcnNzT1rm759+5Zvd+vWjczMTACmTJlCmzZtyvdt2rSJ+Pj4s7Y7nZKSEpxOZ6XFk/r0gZYtobAQ3nrLo18tIiJiC48HEafTSVxcXPm2w+EgMDCQgoICl9uEhYWxY8eOUz63atUqcnJyuPnmm8+pXZlJkyYRHh5evsTGxp7TsZ2vgICKqyIzZ3r0q0VERGzh8SASFBRESEhIpfdCQ0M5ePCgy21O9/kDBw5w55138thjj9G4cWOX251o/PjxFBYWli/btm07p2Nzh5QUE0g++cRMhiciIuLLPB5EoqKi2LNnT6X3ioqKCA4OdrnN6T6fmppKy5YteeCBB86p3YlCQkIICwurtHhabCwkJ5v1F1/0+NeLiIh4lMeDSGJiIhkZGeXbeXl5lJSUEBUV5XKbrKwsYmJiyrenTZvGihUrWLhwIQEBAS63q63uuce8vvoqFBfbW4uIiEhN8ngQ6dWrF06ns/wR27S0NPr3709gYCD79+/n+PHjp7QZMmQIixYtIjs7m+LiYqZOncqAAQMA0y9kzJgxvPrqq9SvX5/i4mIO/feRk6FDhzJt2jTy8/PZtWsXs2fPLm9Xm11xBbRtC04nnDBkioiIiM+xpY/IrFmzSE1NpVGjRixdupSnnnoKgMjISLKzs09p06lTJ0aPHk3Xrl2JiYkhMDCQkSNHAjB16lRKSkq44ooraNiwIQ0bNuTKK68EIDk5mX79+hEfH09cXBydO3fmuuuu89zBVlNAAPz5z2Z92jSNtCoiIr7LYbkyrGkN2LlzJ5mZmSQlJREdHe1Sm9zcXPLz8+ndu3eVfT1OtmbNGg4cOEDv3r1xOBwut3M6nYSHh1NYWOjx/iL79kFMDJSUQEYGJCV59OtFRETOi6u/obYFEW9gZxABMyvva6/Bn/5kXkVERLyFq7+hmvSuFvvv3SfeeAP27rW3FhERkZqgIFKLdesGnTub2zP/7dsrIiLiUxREajGHo+KqyIsvQmmpvfWIiIi4m4JILTdsGISHw48/wooVdlcjIiLiXgoitVz9+qbTKsD06fbWIiIi4m4KIl6gbEyRd9+Fn3+2txYRERF3UhDxAhdfDH36mD4impVXRER8iYKIlyibf2bWLDhyxN5aRERE3EVBxEv88Y/QrBns2gVLlthdjYiIiHsoiHiJOnXgzjvNujqtioiIr1AQ8SJ33QWBgfDpp7Bhg93ViIiInD8FES/SogUkJ5v1F1+0txYRERF3UBDxMmUjrb76Kjid9tYiIiJyvhREvEy/fnDRRVBUZJ6gERER8WYKIl4mIAAeeMCsP/ccHD1qazkiIiLnRUHEC918MzRpAtu2weLFdlcjIiJSfQoiXig0FO6916z/859gWfbWIyIiUl0KIl5q5EgTSL75xjzOKyIi4o0URLxUo0Zw661m/dlnbS1FRESk2hREvNhf/gIOh5mV97vv7K5GRETk3CmIeLELL4RrrzXrU6bYW4uIiEh1KIh4ubFjzetrr8Hu3fbWIiIicq4URLxcjx7QrRuUlEB6ut3ViIiInBsFES/ncFRcFZk2DQ4etLceERGRc6Eg4gMGD4bWrWHvXnOLRkRExFsoiPiAoCDzBA2YTqulpfbWIyIi4ioFER9x++0QEQGbN5vHeUVERLyBgoiPaNAA7r7brP/zn/bWIiIi4ioFER9y331Qpw58/jmsWWN3NSIiImenIOJDYmJg2DCzrmHfRUTEGyiI+JgHHjCvb74JW7faWoqIiMhZKYj4mI4d4YorzJMzzzxjdzUiIiJVUxDxQQ8/bF5nzYL8fHtrERERqYqCiA+6/HLo1QuOHIGnn7a7GhERkTNTEPFREyea15kz4Zdf7K1FRETkTBREfFTfvnDZZXD4sMYVERGR2ktBxEc5HBVXRaZPh1277K1HRETkdBREfNgf/gDdusGhQxpXREREaicFER/mcMBjj5n19HTYs8feekRERE6mIOLjrrwSunSBgwdh8mS7qxEREalMQcTHndhX5P/+D/bts7ceERGREymI+IHkZLj0Uiguhuees7saERGRCgoifuDEqyJTp0JBgb31iIiIlFEQ8RODBkFCAjid8PzzdlcjIiJiKIj4iYAAmDDBrD/3HBQW2lqOiIgIoCDiV4YMgQ4dTAh54QW7qxEREVEQ8SsBAfDoo2Z98mQoKrK3HhEREQURP3PDDdC+vemw+n//Z3c1IiLi7xRE/ExgYMVVkX/+U31FRETEXgoifuimm+Dii+HXX+Hpp+2uRkRE/JmCiB8KCoJJk8z6lCmwY4e99YiIiP9SEPFT114Ll11mZuZ9/HG7qxEREX+lIOKnHI6K2zKzZ8PGjfbWIyIi/klBxI/16GFGXC0thYcftrsaERHxRwoifi4tzYwvsmQJZGTYXY2IiPgbBRE/16ED3HabWX/wQbAse+sRERH/YksQycnJITExkcjISMaNG4flwq/f4sWLadWqFc2bN2fhwoWn3d+nT59K71mWRUREBA6Ho3z5xz/+4bbj8BWPPw6hofD55/Dee3ZXIyIi/sTjQaSkpITk5GS6dOnC2rVryc3N5ZVXXqmyTU5ODsOHD2fChAksX76ciRMnsmnTpvL9y5cvZ8SIEacEmu+//56IiAgKCgrKl3HjxtXEYXm1Fi1g9Giz/tBDcPy4vfWIiIj/8HgQWbZsGYWFhUyePJm2bduSlpbG7Nmzq2wza9Ys+vTpQ0pKCgkJCaSmpjJ37lwAtmzZQmpqKvfee+8p7dasWUP37t2JiIgoX0JCQmrkuLzdgw9CZCRs2AD/PbUiIiI1zuNBZN26dSQlJVGvXj0AOnbsSG5u7lnb9O3bt3y7W7duZGZmAhAdHc2aNWvo0KHDKe1Wr17N6tWriYiIoEmTJjz66KNV3gYqKSnB6XRWWvxFZGTFkzMTJpjxRURERGqax4OI0+kkLi6ufNvhcBAYGEhBQYHLbcLCwtjx3+FAIyMjiYiIOG27zZs3k5ycTFZWFgsWLODFF1/k9ddfP+P3TJo0ifDw8PIlNjb2HI/Ou6WmQmwsbN8O6el2VyMiIv7A40EkKCjolNsjoaGhHDx40OU2Z/t8mWXLlvHcc88RFxdH//79GTVqFIsXLz7j58ePH09hYWH5sm3bNheOyHeEhsLf/27W09LMDL0iIiI1yeNBJCoqij179lR6r6ioiODgYJfbnO3zZ9KkSRPy8/PPuD8kJISwsLBKi7+5+Wa45BITQp56yu5qRETE13k8iCQmJpJxwshZeXl5lJSUEBUV5XKbrKwsYmJiqvyeQ4cOkZCQwKETOjtkZGTQqlWr86je9wUGwpNPmvXnnwc/uygkIiIe5vEg0qtXL5xOJ3PmzAEgLS2N/v37ExgYyP79+zl+mmdHhwwZwqJFi8jOzqa4uJipU6cyYMCAKr+nbt26NG3alJEjR7J27VqmTJnCggULuOeee2rkuHzJVVdBr15w+LB5mkZERKSm2NJHZNasWaSmptKoUSOWLl3KU/+9BxAZGUl2dvYpbTp16sTo0aPp2rUrMTExBAYGMnLkyLN+18svv8xPP/3E73//e2bMmMGiRYvo3bu324/J1zgc8Nxz5nXhQvjsM7srEhERX+WwXBnWtAbs3LmTzMxMkpKSiI6OdqlNbm4u+fn59O7du1p9RM6V0+kkPDycwsJCv+wv8uc/w4wZ0LEjZGZCUJDdFYmIiLdw9TfUtiDiDfw9iOzbB/HxpuNqejq4cBFKREQEcP03VJPeyRlFR0PZ1DyPPgp799pbj4iI+B4FEanS3XdDp07mqsijj9pdjYiI+BoFEalSYCBMnWrWZ86ErCx76xEREd+iICJn1asXDBsGlgX33WdeRURE3EFBRFzyzDNQvz785z8wf77d1YiIiK9QEBGXxMTAI4+Y9f/9XygqsrceERHxDQoi4rIxY6BdO/jll4qnaURERM6Hgoi4LCTEjLgKMGUKbNpkazkiIuIDFETknFx9tVmOHoX771fHVREROT8KInLOpkyB4GD44AN49127qxEREW+mICLnLD7e9BcBGDUKiovtrUdERLyXgohUyyOPQKtWsHUrTJxodzUiIuKtFESkWho0gBdfNOvPPw+rV9tbj4iIeCcFEam2gQPh5puhtBRSUuDIEbsrEhERb6MgIudlyhRo1Aiys+Hpp+2uRkREvI2CiJyXRo3MrRmAv/8dNm60tx4REfEuCiJy3oYNg6uuMrdm7rzT3KoRERFxhYKInDeHA6ZPNx1Yv/gCZsywuyIREfEWCiLiFi1bwqRJZv3BB2H7dnvrERER76AgIm5zzz3QvbuZmfeeezT8u4iInJ2CiLhNYCDMmgV16pih3994w+6KRESktlMQEbfq0MGMugpw332wb5+99YiISO2mICJuN348/OY3sGcPPPCA3dWIiEhtpiAibhccDC+9ZJ6mefVVeP99uysSEZHaSkFEakT37nD//Wb99tvN1REREZGTKYhIjUlLM7dodu0yA53pKRoRETmZgojUmNBQmD/fPEWzdCm8/LLdFYmISG2jICI1qlMneOIJsz56NPzwg731iIhI7aIgIjVuzBjo3RsOHICbb4Zjx+yuSEREagsFEalxgYHw2msQHg5ffVUxFLyIiIiCiHhEy5aQnm7W//pXWL3a3npERKR2qFYQOXLkCC+99BKlpaXs3buX+++/n9TUVHbu3Onu+sSH/M//wI03wvHj5hbNgQN2VyQiInarVhC55ZZbmDlzJgCjR48mNzeXzZs3M2LECLcWJ77F4YDp06FFC/j+exg71u6KRETEbg7LOvfRHcLCwsjKyqJ169Y0adKErVu3UlhYyEUXXURxcXFN1GkLp9NJeHg4hYWFhIWF2V2Oz/j4Y+jf36y/8w5cc4299YiIiPu5+htarSsiDRs2ZOfOnXz66ae0bduWhg0b8vPPPxMeHl7tgsV/9OtnnqQBuOMO2L3b3npERMQ+1QoiY8eO5fLLL2fgwIHcc889rF+/nuuuu44777zT3fWJj3riCUhIMCHkttugtNTuikRExA7VujUDsGnTJkJCQmjdujW//PILOTk5XHHFFe6uz1a6NVOz1q+H3/0ODh82w8GPH293RSIi4i6u/oZWO4j4AwWRmjd7NqSkQECA6Tty+eV2VyQiIu5Qo31E9u3bxyOPPMLx48fJy8vjj3/8I9dccw3fffddtQsW/3T77TBihLk1c9NN8MsvdlckIiKeVK0gMnz4cNavX4/D4WDUqFFERETQqFEj7rjjDnfXJz7O4YBp00x/kV27YNgwDQEvIuJPqnVrpkGDBuTm5tKsWTOaNm3Krl272Lt3L/Hx8RzwoVGqdGvGczZtgq5dobjY9BVJS7O7IhEROR81emumSZMmfP3117zxxhtccsklBAcHk52dTdOmTatdsPi39u1NfxEwc9G8+6699YiIiGcEVafRE088wc0330ydOnVYtGgRq1evZvDgwUyePNnd9YkfueEG+OILeOEFuOUW+OYbaN3a7qpERKQmVfupmeLiYoKCgggNDaWgoIDdu3fTvn17d9dnK92a8bwjR6BnTzMpXmIifP45hITYXZWIiJyrGr01A6afiNPpZO3atRw7dsznQojYIzgY3ngDoqJgzRp44AG7KxIRkZpUrSBSWFjI4MGDadasGT179qRZs2YMHToUp9Pp7vrED7VqBfPmmfX0dHj9dXvrERGRmlOtIHLvvfdSWlrK9u3bOXToENu2bePYsWOMHDnS3fWJn7rySnj4YbOekgI5OfbWIyIiNaNafUSio6PJzMyk9Qk9CfPy8ujSpQu//vqrO+uzlfqI2OvYMRgwAFauNJ1WV6+Gxo3trkpERFxRo31EWrZsycqVKyu9t3LlSlq1alWdf07ktIKCTH+Rtm1h61YYOtR0ZhUREd9Rrcd3n3/+ea6++mreeOMN2rRpw48//siXX37Je++95+76xM9FR8M770BSEnz2GYwcCS+9ZEZkFRER71etKyK9evXiu+++4/LLL8fhcNCnTx9yc3OpX7++u+sT4eKLYdEiMzHe7Nnw/PN2VyQiIu7ittl38/PzadmyJcePH3fHP1crqI9I7TJlCowZYwLJu++aDq0iIlI71fg4Iqfjpkwjclr33w933FExU68mexYR8X5uDSIO3biXGlQ2U2/PnuB0QnIy7Ntnd1UiInI+3BpERGpacDD861/mcd4ffoDrr4ejR+2uSkREqsvlp2Y6d+5c5RWPI3quUjykcWPzJE337rBqFYwaZa6U6IKciIj3cTmI3H///W770pycHG677Ta2bNlCSkoKTz/99Flv6yxevJgHHniAo0eP8uyzzzJs2LBT9qenp7Nq1apK76enp/O3v/2N+vXrM2vWLPr27eu24xD7XHIJLFgAgwbBiy/CRRfB6NF2VyUiIufM8rDDhw9brVu3tu6++25ry5Yt1lVXXWW9/PLLVbbJzs62goODrZdeeslav3691a5dO2vjxo3l+z/44AOrXr16Vu/evSu1++CDD6zQ0FDrrbfesv7zn/9YcXFx1t69e12utbCw0AKswsLCczpG8Zynn7YssCyHw7Jef93uakREpIyrv6Ee7yOybNkyCgsLmTx5Mm3btiUtLY3Zs2dX2WbWrFn06dOHlJQUEhISSE1NZe7cuQBs2bKF1NRU7r333lPaTZ8+nREjRjBo0CAuu+wyBg0axJIlS2rkuMQeY8fCvfeCZcGf/mSGgxcREe/h8SCybt06kpKSqFevHgAdO3YkNzf3rG1OvKXSrVs3MjMzATPvzZo1a+jQocM5tTudkpISnE5npUVqN4fDDHBWNvz7H/8IWVl2VyUiIq7yeBBxOp3ExcWVbzscDgIDAykoKHC5TVhYGDt27AAgMjKSiIiIc253OpMmTSI8PLx8iY2NdfWwxEaBgTB3Llx+ORQVmYHOfvzR7qpERMQVHg8iQUFBhISEVHovNDSUgwcPutzmbJ+vbrvx48dTWFhYvmzbtu2s3yG1Q2govPUWdOoEu3aZWXt377a7KhERORuPB5GoqCj27NlT6b2ioiKCg4NdbnO2z1e3XUhICGFhYZUW8R7h4bBsmRljZMsWuPpqKC62uyoREamKx4NIYmIiGRkZ5dt5eXmUlJQQFRXlcpusrCxiYmLO+btcbSfe64ILYPlyaNQI1q6FIUNM3xEREamdPB5EevXqhdPpZM6cOQCkpaXRv39/AgMD2b9//2knzRsyZAiLFi0iOzub4uJipk6dyoABA876XUOHDmXatGnk5+eza9cuZs+e7VI78W4XXgjvvQf16sGKFXD77WZ+GhERqX1s6SMya9YsUlNTadSoEUuXLuWpp54CTMfT7OzsU9p06tSJ0aNH07VrV2JiYggMDGTkyJFn/a7k5GT69etHfHw8cXFxdO7cmeuuu87txyS1T7duZij4oCCYPx/GjTOP+IqISO3isCx7/ud5586dZGZmkpSURHR0tEttcnNzyc/Pp3fv3i71ESmzZs0aDhw4QO/evc9pYj5XpzCW2mvuXLjlFrM+cSL89a/21iMi4i9c/Q21LYh4AwUR3zB1asXw73//Ozz6qL31iIj4A1d/QzX7rvi8UaPgmWfM+oQJ8N87gSIiUgsoiIhfGDsW0tLM+kMPweTJ9tYjIiKGgoj4jfHjK/qIPPCAuWUjIiL2UhARvzJxYkUfkdGjYfp0e+sREfF3CiLid/72N3jwQbM+ciS89JK99YiI+DMFEfE7DgdMmgRjxpjtu++GV16xtSQREb+lICJ+yeGAf/7TPFFjWWb01f8O9isiIh6kICJ+y+GA556De+6pCCPqwCoi4lkKIuLXHA5IT6+4TTN6tBn0TMP8iYh4hoKI+L2y2zRlj/ZOnKi5aUREPEVBRAQTRiZONLdqAJ59Fu66C04zGbSIiLiRgojICUaPhtmzISAAZs2C4cPhyBG7qxIR8V0KIiInuf12eP11qFPHvA4eDIcO2V2ViIhvUhAROY2hQ+Htt6FuXXj/fRg4EJxOu6sSEfE9CiIiZzBwICxfDmFh8Nln0KcP7Nxpd1UiIr5FQUSkCj17wqpV0KgRfPMNJCXBhg12VyUi4jsURETO4re/hYwMiI+Hn36CHj1g5Uq7qxIR8Q0KIiIuaNfOhJEePaCw0Ny2ee01u6sSEfF+CiIiLoqOho8+ghtvhKNHYcQIM5OvBj4TEak+BRGRcxAaCgsWwEMPme3HHoPbbtNYIyIi1aUgInKOAgJg0iSYMQMCA+HVV+HKK2H/frsrExHxPgoiItV0113w7rvQoIHpvNqjB+Tl2V2ViIh3URAROQ8DB8IXX0BMDOTmQteuph+JiIi4RkFE5Dx16gRffQWJifDrrzBggJk0T51YRUTOTkFExA1atDCjr956K5SWwtixcPPNcPCg3ZWJiNRuCiIibhIaCi+/DC+8AEFB5umaHj1g61a7KxMRqb0URETcyOGA1FT4+GNo3Bi+/db0G9FIrCIip6cgIlIDevWCzEzo0gX27YMrroApU9RvRETkZAoiIjUkNhY+/xxuucX0GxkzBoYPh6IiuysTEak9FEREalDduvDKK/D882bws4ULzVWSb7+1uzIRkdpBQUSkhjkcMGoUfPqpebrm++8hKQnS03WrRkREQUTEQ3r0MFdCkpOhpMR0ah06VEPDi4h/UxAR8aDoaFi61HRcrVMH/v1v6NwZvv7a7spEROyhICLiYQ4H3H8/fPkltGljxhn5/e/NaKylpXZXJyLiWQoiIjbp2hW++QZuuAGOHTOjsSYnw+7ddlcmIuI5CiIiNgoPh0WL4MUXISQE3n8fLrkEliyxuzIREc9QEBGxmcMBd98Nq1dDx46wZw9cdx2MGKGOrCLi+xRERGqJjh1NGHnoIQgIgNdeg4QE+OgjuysTEak5CiIitUhICEyaZEZkbdsWtm83w8OPGqWZfEXENymIiNRCl11mxhy55x6z/cILesxXRHyTgohILdWgAUybBh98AM2bw+bNJqA8/DAcOmR3dSIi7qEgIlLLDRgAOTlmwrzSUnPrpmNHWLXK7spERM6fgoiIF4iMhHnzzGO9zZvDli3Qty/cfjv8+qvd1YmIVJ+CiIgX+eMfITe3ou/InDlw8cVmLBJNoCci3khBRMTLhIebviNffAEdOpiRWIcNg6uvhp9+srs6EZFzoyAi4qV69DBDxP/1rxAcDMuWmWAyZYoZMl5ExBsoiIh4sZAQmDgR1q2Dnj3NWCNjxphHfT/5xO7qRETOTkFExAdcdJEJHjNnQlSUecqmTx+48UbYts3u6kREzkxBRMRHBATAnXea8UZGjjTbb7wB7dvDP/4Bhw/bXaGIyKkURER8THQ0pKdDZib8/vdm8LMJE+A3v4G339bTNSJSuyiIiPioSy+Fzz6D+fPN2CM//giDBsFVV8HGjXZXJyJiKIiI+DCHA/7nf2DTJnjwQahTxwwZf8klZiySnTvtrlBE/J2CiIgfaNAAnnwSNmyA5GQ4fhxefBHatTOP/xYX212hiPgrBRERPxIfb/qJfPopdOsGBw7A44+bQDJjhsYfERHPUxAR8UO9esFXX5mnatq2hV274M9/hoQEWLpUHVpFxHNsCSI5OTkkJiYSGRnJuHHjsFz4X73FixfTqlUrmjdvzsKFCyvtS09Pp2nTprRp04aVK1eWv79v3z4cDkelZd68eW4/HhFv5HDA9debuWumTjVP22zcaOaz6dVLA6KJiGd4PIiUlJSQnJxMly5dWLt2Lbm5ubzyyitVtsnJyWH48OFMmDCB5cuXM3HiRDZt2gTA8uXLGTt2LDNnzmTevHmkpKSwb98+ADIzM+nZsycFBQXly4033ljThyjiVYKD4b774IcfYPx4CA0189j06QP9+pl1EZGa4vEgsmzZMgoLC5k8eTJt27YlLS2N2bNnV9lm1qxZ9OnTh5SUFBISEkhNTWXu3LkATJ8+nREjRjBo0CAuu+wyBg0axJIlSwBYs2YNPXr0ICIionypU6dOjR+jiDcKD4e0NNiyxQyIVqcOrFxpho4fMMDcyhERcTePB5F169aRlJREvXr1AOjYsSO5ublnbdO3b9/y7W7dupGZmXnWfatXr+bf//43DRo0IDY2lhdeeMHdhyPic2JizIBoW7bAXXdBUBCsWAHdu5sZfteutbtCEfElHg8iTqeTuLi48m2Hw0FgYCAFBQUutwkLC2PHjh1n3bdlyxZuvfVWcnNzmTx5Mv/7v//LV1X837qSkhKcTmelRcRftWxpnqTZvBluvx0CA+H99yEx0QyMpkAiIu7g8SASFBRESEhIpfdCQ0M5ePCgy21O/HxV+zZs2MD48eNp2bIl119/PcOGDWPx4sVn/J5JkyYRHh5evsTGxlbrGEV8SVwczJ4N330Hf/qTmcPm7bdNIPnDH2DVKj1lIyLV5/EgEhUVxZ49eyq9V1RURHBwsMttTvx8VftO1qRJE/Lz88/4PePHj6ewsLB82aZpS0XKxcfDa6+Zp2z+9CdzheTDD6FvX3PbZulSKC21u0oR8TYeDyKJiYlkZGSUb+fl5VFSUkJUVJTLbbKysoiJialy308//USPHj0qPRqckZFBq1atzvg9ISEhhIWFVVpEpLL27U0gKevUGhoKX39tHvtNSDD7jh61u0oR8RYeDyK9evXC6XQyZ84cANLS0ujfvz+BgYHs37+f48ePn9JmyJAhLFq0iOzsbIqLi5k6dSoDBgwAYOjQoUybNo38/Hx27drF7NmzGTBgAC1btmT37t08/PDDZGZm8vDDD/PVV19xxx13ePR4RXxV69amU+vWrfDQQxAWZq6WjBhhrp688IKGjhcRF1g2WLp0qVWvXj0rOjraaty4sbVhwwbLMpcurKysrNO2efjhh63g4GArLCzM6tKli3Xw4EHLsiyrtLTUuvnmm626detadevWta655hqrtLTUsizLWrdundWlSxcrODjY6ty5s/XJJ5+cU52FhYUWYBUWFlb/YEX8xP79ljVpkmU1aWJZpteIZUVEWNa4cZb10092Vycinubqb6jDsuzpZrZz504yMzNJSkoiOjrapTa5ubnk5+fTu3fvU/qBrFmzhgMHDtC7d28cDodbanQ6nYSHh1NYWKjbNCIuOnQI5syB556D77837wUGwpAhcP/9pj+JiPg+V39DbQsi3kBBRKT6SkvN475TppiB0cr87nfwl7/AddeZQdNExDe5+huqSe9EpEYEBMA118DHH8O338Jtt5nh5L/+Gm66Cdq0gSeegJ077a5UROykICIiNa5TJ3j5Zfj5Z3j8cWjSBLZvh0cfhdhYuPFGM8mers+K+B8FERHxmKZN4bHH4KefzGO+3bvDsWPwxhtmkr0OHeD552H/frsrFRFPURAREY8LDTWDon35pbltc/fdUL8+bNxoOrQ2bw533AGrV+sqiYivUxAREVt16gQvvgg7dphxSS65xDx58/LLpmNrx46mw+tJAzKLiI9QEBGRWiEszIzUun49fPEFDB9urpzk5MCYMeYqyZAh8O675naOiPgGPb5bBT2+K2Kv/fth0SJzdWTNmor3L7gAbrnFPInTvr1t5YlIFTSOiBsoiIjUHjk5ZqC0uXMr36bp1s1cPbnxRtMZVkRqBwURN1AQEal9jhyB994zV0mWLYOy6akCA6F/fxNKBg+GBg3srVPE3ymIuIGCiEjttns3vP46zJ9vBkorU7eumQ14+HD4wx80gquIHRRE3EBBRMR7bNkCCxbAvHkVc9wAREWZUHLDDdC3r0KJiKcoiLiBgoiI97EsWLvWXCVZtAh27arYVxZKrr8e+vVTKBGpSQoibqAgIuLdjh+Hzz83I7f+61/mVk6ZyEjTl+T6682VkpMm9BaR86Qg4gYKIiK+oyyUvPmmCSUnXilp2BCuuspcLbnySggPt61MEZ+hIOIGCiIivunEULJkCfzyS8W+OnXMvDeDBsG110KLFvbVKeLNFETcQEFExPeVlprB0t56C5Yuhe++q7y/a1cTSK66Cjp3hgCNRy3iEgURN1AQEfE/mzaZQLJ0KWRkVJ50r1kzc+vmqqvgiit0C0ekKgoibqAgIuLfdu2Cd94xA6h9+CEcOFCxLygIfv97uPpqE046dACHw75aRWobBRE3UBARkTIlJaZfyfvvm2CyeXPl/TExZvC0K64wI7w2bmxPnSK1hYKIGyiIiMiZbNlSEUo+/dQElRP99rcmlPzhD9CjB4SE2FOniF0URNxAQUREXHHokLla8uGHsGIFrF9feX/dutCzp3kap08f6NLF3NoR8WUKIm6gICIi1bFzJ3z0kQklH35otk/UsCH06mVCSd++0KmTnsYR36Mg4gYKIiJyviwLcnJg1SqzfPIJ7N9f+TORkdC7t7lq0rOneUxYV0zE2ymIuIGCiIi42/HjsG6dCSUrV5pbOkVFlT9Tvz5cdllFMPnd78ztHRFvoiDiBgoiIlLTjh2DzEzT4fXzz+GLL069YlKnjhlYrUcPE1C6dzdjmojUZgoibqAgIiKeVlpqbuV8/nnFsmPHqZ9r3boilHTvDh07ajZhqV0URNxAQURE7GZZkJdnAklGhlmysyuP+ApQr565atKtW8XSsqUGWRP7KIi4gYKIiNRGTiesXg1ffmmCyVdfnXo7B6BJk8rBJDERoqI8Xq74KQURN1AQERFvUFoKGzeacFK2rFtn+p+cLC7ODLbWpUvFa6NGnq9ZfJ+CiBsoiIiItzp8GL79tnI4+f7703+2ZcuKUNK5M1x6KTRvrts6cn4URNxAQUREfMn+/ZCVZZ7S+eYb83rynDlloqNNIOnUqeL14ovVIVZcpyDiBgoiIuLrnE5z5aQsmHz7LXz3nRnv5GTBwWaW4YQEs1xyiXmNidHVEzmVgogbKIiIiD86fBhyc00o+fZb09/k229NaDmdiIiKUHLJJWbp0EF9T/ydgogbKIiIiBiWBVu3mlCSk2MeIc7JgU2bTn/1BKBxYxNITl6aNtUVFH+gIOIGCiIiIlUrKTFP7JSFk+xsczVl69Yzt4mMhIsuMkv79hXrbdqoD4ovURBxAwUREZHqOXDABJTc3MrLDz+cOhhbmaAgaNvWhJILL4T4eLNceCFccIGuongbBRE3UBAREXGvQ4fMkzqbNpmgcuLrgQNnble/PrRrVzmgtG1rFoWU2klBxA0UREREPMOyID/fhJKNG82YJ99/b0LL1q1n7ocCZmbislBy4tKmDbRqZZ72Ec9TEHEDBREREfsdOWLCyObNFQHl++/NbZ6ffjIjy56Jw2EeL46LO/3SvDkEBnrsUPyKgogbKIiIiNRuR4+aMPLDDxXLli3mNS8PDh6sun1QELRoYWYzbtWq4rVsvUULXVGpLld/Q4M8WJOIiIhb1alj+o60a3fqPsuCPXtMIMnLgx9/rFjPy4Offzbz8WzdeuanfBwO87hxy5YQG1vxeuJ6kya6qnI+dEWkCroiIiLiu44fhx07zBWVsmXr1srbhw+f/d8JCjK3eFq0MLeBWrSoWGJizHLBBRASUuOHVKvo1owbKIiIiPgvy4K9e2HbNnP15OTXn3+GX36puo/KiRo1MoElJsa8lq1fcEHF0rSp74yloiDiBgoiIiJSlWPHYOdO2L7dPPWzffupy44dpsOtKxwOE1guuACaNasIKM2amZDSrFnFekRE7X5sWX1EREREalhZZ9cWLc78GcuCX381gSQ/37yWLfn55qrKL7+YQHPsmOnXsmcPrF9f9XcHB1eEk6ZNTV+VsuXk7UaNTK21US0tS0RExDc4HBAdbZaEhDN/rrQU9u2rCCYnLrt2mWXnTvO6f7+5yrJtm1lcqSEy0sz/U7Y0aVJ5u0UL6NnTbYftMgURERGRWiAgoCIUdOxY9WcPH64cTnbvrlh27aq8vXevCTm//mqWTZtO/2/+5jdmziBPUxARERHxMqGhFeOdnM3x4+ZKS9ktn7Jl9+7K23FxNV/36SiIiIiI+LDAwIq+IrVRgN0FiIiIiP9SEBERERHbKIiIiIiIbRRERERExDYKIiIiImIbBRERERGxjYKIiIiI2MaWIJKTk0NiYiKRkZGMGzcOV+bdW7x4Ma1ataJ58+YsXLiw0r709HSaNm1KmzZtWLlyZaV9jzzyCJGRkXTs2JH1Zxu4X0RERDzK40GkpKSE5ORkunTpwtq1a8nNzeWVV16psk1OTg7Dhw9nwoQJLF++nIkTJ7Lpv2PULl++nLFjxzJz5kzmzZtHSkoK+/btA2DGjBnMmDGDt99+m3/84x/cdNNNHHF1CkQRERGpcR4PIsuWLaOwsJDJkyfTtm1b0tLSmD17dpVtZs2aRZ8+fUhJSSEhIYHU1FTmzp0LwPTp0xkxYgSDBg3isssuY9CgQSxZsqR839ixY+nZsyfXXnst7du357PPPqvxYxQRERHXeDyIrFu3jqSkJOrVqwdAx44dyc3NPWubvn37lm9369aNzMzMKvdZlkV2dvYZ24mIiIj9PD7XjNPpJO6EmXUcDgeBgYEUFBQQGRnpUpuwsDB27NhR5b7i4mJKS0tP2bd58+Yz1lZSUkJJSUn5dmFhYfl3iIiIiOvKfjvP1g/U40EkKCiIkJCQSu+FhoZy8ODBMwaRk9uUfb6qfUFB5tDO1O50Jk2axF//+tdT3o+NjXXhyERERORkRUVFhIeHn3G/x4NIVFQUOTk5ld4rKioiODi4yjZ79uw57efPtK9u3brUrVuXPXv2EBYW5tL3jB8/njFjxpRvl5aW8uuvvxIdHY3D4Ti3Az0Dp9NJbGws27ZtK69Lzo/OqXvpfLqfzql76Xy6X02cU8uyKCoqonnz5lV+zuNBJDExkZdeeql8Oy8vj5KSEqKioqpsk5GRwR133AFAVlYWMTExlfb169fvlH1du3YlIyODtm3blu9r3779Gb8nJCTklKs1ERER536QLggLC9N/QG6mc+peOp/up3PqXjqf7ufuc1rVlZAyHu+s2qtXL5xOJ3PmzAEgLS2N/v37ExgYyP79+zl+/PgpbYYMGcKiRYvIzs6muLiYqVOnMmDAAACGDh3KtGnTyM/PZ9euXcyePbvSvqeeegqn08nmzZtZvHhx+T4RERGxn8eDSFBQELNmzSI1NZVGjRqxdOlSnnrqKQAiIyPJzs4+pU2nTp0YPXo0Xbt2JSYmhsDAQEaOHAlAcnIy/fr1Iz4+nri4ODp37sx1110HwN13302TJk1o0aIFCQkJ3HrrrXTp0sVzBysiIiJV8vitGYBrr72WH374gczMTJKSkoiOjgaq7ln7xBNPMHz4cPLz8+ndu3d5Xw+Hw8HcuXMZNWoUBw4coHfv3uX9OUJCQvjwww/5z3/+Q0hICN26dav5gzuLkJAQHnvssVNuAUn16Zy6l86n++mcupfOp/vZeU4dlivjq4uIiIjUAE16JyIiIrZREBERERHbKIiIiLjZ/v37+frrrykoKLC7FJFaT0HEg3JyckhMTCQyMpJx48adddhbOb29e/cSFxfH1q1by9/Tua2+pUuX0qZNG4KCgrj00kv57rvvAJ3T6nrzzTdp3bo1KSkptGjRgjfffBPQ+XSHgQMHls/W/umnn3LxxRfTqFEjJk+ebG9hXmbUqFE4HI7ypV27doB9f6MKIh5SUlJCcnIyXbp0Ye3ateTm5pb/ByWu27t3L9dcc02lEKJzW30//PADt912G08++ST5+flceOGFpKSk6JxWU2FhISNHjuSzzz4jOzub9PR0xo0bp/PpBvPnz2f58uUA7Nmzh2uvvZZhw4aRkZHB/PnzWbVqlc0Veo+1a9fy3nvvUVBQQEFBAVlZWfb+jVriEUuWLLEiIyOtAwcOWJZlWd9++63Vo0cPm6vyPv369bOef/55C7Dy8vIsy9K5PR/vvPOONWPGjPLtlStXWnXr1tU5raaff/7ZmjdvXvn2unXrrAYNGuh8nqd9+/ZZTZs2tdq3b2/NmTPHmjJlinXRRRdZpaWllmVZ1ltvvWUNHz7c5iq9w9GjR62wsDCrqKio0vt2/o3qioiHrFu3jqSkJOrVqwdAx44dyc3Ntbkq7/PSSy8xatSoSu/p3FbfNddcw1133VW+vWnTJuLj43VOqyk2Npbhw4cDcPToUaZMmcLgwYN1Ps/TAw88wODBg0lKSgLMf/N9+vQpHzOqW7duZGZm2lmi18jOzqa0tJRLL72UunXrMnDgQH7++Wdb/0YVRDzE6XQSFxdXvu1wOAgMDFRntnN04jkso3PrHkeOHOHZZ5/lz3/+s87peVq3bh3NmjXjgw8+YOrUqTqf52HVqlV8/PHHPP300+XvnXw+w8LC2LFjhx3leZ3c3Fzat2/P3LlzWb9+PUFBQdx11122/o0qiHhIUFDQKSPWhYaGcvDgQZsq8h06t+7x2GOPUb9+fVJSUnROz1PHjh1ZsWIF8fHxOp/n4fDhw9x9991Mnz6dhg0blr9/8vnUuXTd8OHDWbt2Ld27dyc+Pp5p06bx4YcfUlpaatvfqIKIh0RFRbFnz55K7xUVFZUPVS/Vp3N7/lauXEl6ejoLFiygTp06OqfnyeFw0KVLF1599VX+/e9/63xW09///ncSExO5+uqrK71/8vnUuay+Jk2aUFpaSrNmzWz7G1UQ8ZDExEQyMjLKt/Py8igpKSEqKsrGqnyDzu35ycvLY9iwYaSnp9OhQwdA57S6Pv30U8aNG1e+HRwcjMPh4OKLL9b5rIYFCxawdOlSIiIiiIiIYMGCBYwcOZJXX3210vnMysoiJibGxkq9x7hx41iwYEH5dkZGBgEBASQkJNj3N+qRLrFiHT161GrcuLH18ssvW5ZlWSkpKdY111xjc1XeixOemtG5rb6DBw9aHTp0sO68806rqKiofDly5IjOaTXs2LHDCgsLs2bMmGH9/PPP1i233GINHDhQf6PVtG3bNisvL698GTJkiPXMM89Ye/bssUJDQ60PP/zQOnLkiDVw4EArNTXV7nK9wty5c624uDjro48+spYvX25deOGF1q233mrr36iCiActXbrUqlevnhUdHW01btzY2rBhg90lea0Tg4hl6dxW11tvvWUBpyx5eXk6p9W0YsUKq0OHDlbDhg2toUOHWrt377YsS3+j7jBixAhrzpw5lmVZ1vTp0606depYkZGRVlxcnLVz5057i/MiDz30kBUeHm5FRUVZo0aNsoqLiy3Lsu9vVLPvetjOnTvJzMwkKSmJ6Ohou8vxKTq37qdz6l46n+6Vl5fHxo0b6dmzJw0aNLC7HJ9gx9+ogoiIiIjYRp1VRURExDYKIiIiImIbBRERERGxjYKIiIiI2EZBRERERGyjICIitconn3xCRESE3WWIiIcoiIiIV2jdujWffPKJ3WWIiJspiIiIiIhtFEREpFYbOHAgDoeDn376iT59+uBwOHjyySfL93/wwQckJCQQERFBSkoKJSUl5ftat27NRx99xPjx42nWrBkbNmyw4xBEpAoKIiJSq/3rX/+ioKCA2NhY3nnnHQoKCvjLX/4CwJYtWxg0aBCjR49mzZo1rF69mmeeeaZS+wkTJrB9+3bmz59P69atbTgCEamKgoiI1Gr169cnIiKCgIAAGjRoQEREBCEhIQC8/vrrXHrppaSkpBAfH8/IkSN5++23K7UPDw9n7ty59OvXj/r169txCCJShSC7CxARqa7t27eTlZVV/pTNsWPHTpn87L777rOhMhFxlYKIiHiFgIAATp6js0WLFiQnJ/Pss88CcPz4cQ4ePFjpM7oKIlK76daMiHiFtm3bsmLFCn755Rc+/vhjAG666SY+//xzvv/+e0JCQnjhhRe47bbbbK5URM6FgoiIeIVnnnmG9957j5YtW/L4448DJpy89tprjBkzhnbt2rF+/XoWLlxob6Eick4c1snXOkVEREQ8RFdERERExDYKIiIiImIbBRERERGxjYKIiIiI2EZBRERERGyjICIiIiK2URARERER2yiIiIiIiG0URERERMQ2CiIiIiJiGwURERERsc3/A8f1S7/Snr00AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 600x400 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "plt.rcParams['font.sans-serif']=['SimHei']\n",
    "plt.rcParams['axes.unicode_minus']=False\n",
    "\n",
    "# 1，参数初始化\n",
    "# 1.1，隐含层输入\n",
    "x=tf.constant(1.,dtype=tf.float32)\n",
    "# 1.2，标签\n",
    "y=tf.constant(0.8,dtype=tf.float32)\n",
    "# 1.3，学习率\n",
    "lr=tf.constant(0.5,dtype=tf.float32)\n",
    "\n",
    "# 1.4，隐含层参数\n",
    "wh=tf.Variable(0.2,dtype=tf.float32)\n",
    "bh=tf.Variable(0.1,dtype=tf.float32)\n",
    "# 1.5，输出层参数\n",
    "wo=tf.Variable(0.3,dtype=tf.float32)\n",
    "bo=tf.Variable(0.2,dtype=tf.float32)\n",
    "\n",
    "iter=50\n",
    "losses=[]\n",
    "# 2，训练\n",
    "for i in range(0,iter+1):\n",
    "    with tf.GradientTape(persistent=True) as tape:\n",
    "        # 2.1，前向传播计算预测值\n",
    "        # 隐含层线性输出公式\n",
    "        zh=wh*x+bh        \n",
    "        # 隐含层激活输出公式\n",
    "        yh=tf.sigmoid(zh)        \n",
    "        # 输出层线性输出公式\n",
    "        zo=wo*yh+bo\n",
    "        # 输出层激活输出公式\n",
    "        yo=tf.sigmoid(zo)        \n",
    "        # 2.2，计算损失公式\n",
    "        loss=0.5*(tf.square(y-yo))\n",
    "        \n",
    "    # 将损失添加到列表\n",
    "    losses.append(loss)\n",
    "    # 打印本轮参数和损失\n",
    "    if i%10==0:\n",
    "        print(i,'wh:',wh.numpy(),',bh:',bh.numpy(),',wo:',wo.numpy(),',bo:',bo.\n",
    "numpy(),',yo:',yo.numpy(),',loss:',loss.numpy())\n",
    "\n",
    "    # 2.3，求Loss对输出层参数的偏导数\n",
    "    dLoss_dwo,dLoss_dbo=tape.gradient(loss,[wo,bo])\n",
    "    # 2.4，更新输出层参数\n",
    "    wo.assign_sub(lr*dLoss_dwo)\n",
    "    bo.assign_sub(lr*dLoss_dbo)\n",
    "    \n",
    "    # 2.5，求Loss对隐含层参数的偏导数\n",
    "    dLoss_dwh,dLoss_dbh=tape.gradient(loss,[wh,bh])\n",
    "    # 2.6，更新隐含层参数\n",
    "    wh.assign_sub(lr*dLoss_dwh)\n",
    "    bh.assign_sub(lr*dLoss_dbh)    \n",
    "    \n",
    "# 3，可视化损失\n",
    "plt.figure(figsize=(6,4))\n",
    "plt.plot(losses,color=\"b\")\n",
    "plt.xlabel(\"Iter\")\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3b3e36e-8418-4caf-af51-d8ac80e2a321",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
